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Probability of an event: a set of outcomes that is the subset of the sample space. 








e aratio that represents the likelihood of an event 


o scientific way to measure uncertainty or randomness 
— e.g. outcome- of tossing a coin; rolling a die; drawing a card; 
— e.g. chance of- winning the lottery; defaulting on a loan; committing of fraud 


# of times 
event happens 









Probability = No. of Favourable Outcome 
Total Number of Outcomes 


# of trials 








Marginal probability: the probability of an event occurring, P(A) 


e thought of as an unconditional probability, not conditioned on another event. 
o e.g. the probability that a card drawn is Red, P(Red) = 26/52 = 0.5 
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Intersection Union 


(and) (or) Conditional 


| Mutually Inclusive 


Mutually Exclusive 





P(A and B) = P(A) x P(B) 





P(ANB) 
PUB) 


P(A|B) = 





P(A or B) = P(A) + P(B) (P(A or B) = P(A) + P(B) - P(A and B) 
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Two Events Occuring 


e probability of drawing a 9 and a King 
e P(9 and K) = P(9 N K) = P(9) * 


= 4/52 * 4/52 = 16/ 2,704 

















P(K) 
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Mutually Exclusive Events 


Probability of drawing a 9 or King 
P(9 or K) = P(9) + P(K) 
= 4/52 + 4/52 = 8/ 52 











Mutually Exclusive Events 


A 


P(A or B) = P(A) + P(B) 


B 
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Non-Mutually Exclusive Probability 





e probability of drawing a 9 or a Red card 
e P(Red or 9) 

= P(Red) + P(9) - P(Red N 9) 

= 26/52 + 4/52 - (1/2 * 4/52) 

= 28/52 


Non-Mutually Exclusive Events 


A B 


P(A or B) = P(A) + P(B) — P(A and B) 











the probability of event A, given that B occurred, P(A|B) 
e derived from the probability multiplication rule, P(A and B) = P(A)*P(BJA) 
o given that you drew a Face card, what's the probability that it’s a King 


P(A|B) = = 














Dependent Probability 
e P(K | Face ) = P(KNFace) / P(Face) 
= 4/52 | (12/52) 
= 4/12 = 1/3 


























e the total probability of an outcome realized via several distinct events 
o a fundamental rule relating marginal probabilities to conditional probabilities 
o total probability of event A as a sum of conditional probabilities Bi events 


o used to find the probability of an event, P(A), when P(A) cannot be calculated directly 
Q 





P(A) => P(AN By) 


P(A) = )_ PAIRA) P(B:) 


P(A) = YP(ANB,) => P(A]JB,)“P(B,) 








P(A) = YP(ANB,) = 0 + P(ANB2) + P(ANB3) + P(ANB4) + P(ANBs) + 0 


Bayes Theorem 





e Studied logic & theology, 1719, Univ. of Edinburgh 


e Motivated to rebut David Hume's argument 
o against believing in miracles on the evidence of testimony 


e Findings on probability theory unpublished 


o were passed to his friend Richard Price after his death (1761) 





e Solution to inverse probability problem 
o published in the Philosophical Transactions of the RSL, 1973 
o popularized by Pierre-Simon Laplace, 1814 
e Use of a uniform prior distribution for a binomial parameter 
o not general postulate,but now in all conditioned probabilities like SE, PML, RA 
o simultaneous localization & mapping, regularization, or information theory 





e Spam Filtering 
o attack spam by using words and phrases in the messages 
o determine the probability that the message is unwanted 


se 





EE Microsoft 
e Microsoft Operating System 


o handwriting recognition; recommender systems; the question-answering 
o tracking business sales; preload the applications; predict traffic jam 


e Netflix S1-million contest 


o online commerce relies on recommender filters 


o contest illustrates the prominent role of Bayesian 
— Yehuda Koren, Christopher T. Volinsky, & Robert M. Bell 
— winning algorithm had a Bayesian “perspective” 


e Google Searches 
o several forms of Bayes to search through the billions of documents 
o Bayes’ filters through billions of pages, find 2 relevant ones in 0.31s 


e update the probability of an event as per new evidence 
o if the data support the hypothesis then the probability goes up 
o if itdoes not match, then probability goes down 

e adds ‘subjectivity’ to statistical modeling, ML, and Al 


o knowledge discovery and predictive analytic of business, medicine, & technology 
o genetics, linguistics, image processing, brain imaging, epidemiology 





| Posterior distribution is proportional to data distribution * prior distribution | 








e Determine P(A|B), given P(B|A) 
o updating probability with new info 
o derived from rule of multiplication 4 
— if P(AN B) = P(AJB) P(B) / 


— and P(B N A) = P(BIA) P(A) | 
— then P(B N A) = P(AN B) | 


o P(A|B) P(B) = P(BJA) P(A) \ 
— P(A|B) = P(BJA) P(A) / P(B) \ 








Sample Space 





new 
Sample Space 





Bayes’ Theorem 


= X / new evidence 


Likelihood Prior 


How probable is the evidence How probable w 
given that our hypothesis is true? before observing the 


P(e | H) P(H) 
P(H Aa a dirti ti) 


Posterior 


How probable is our hypothesis 
given the observed evidence? 


(Not directly computable) 





Marginal 
probability 
Prior 
P(A) 
Total 
population 


Likelihood 
Conditional Joint 
Probability Probability 


T 
P(B’ |A) HO 
P(B|A') 


Normalization 


Posterior = likelihood x prior / new evidence 





Conditional Probability: the probability of Bayesian Probability: the conditional probability of 
event B+ occurring in the presence of A+ event A+, given that event B+ has already occurred 


P(B+| A+) 





How often does B+ happen when A+ happens? How often does A+ happen after B+ has happened? 


Appendix 


Applying Bayesian Inference 





A certain disease affects about 1 out of 10,000 people. There is a test to check whether 
the person has the disease. The test is quite accurate. 


P(+) (suggesting the person has the disease), given that the person does not have the 
disease, is only 2 %; 


P(-) (suggesting the person does not have the disease), given that the person has the 
disease, is only 1 %. 


A random person gets tested for the disease and the result comes back positive. 


What is the probability that the person has the disease P(D)? 


P(neg) 
0.99 





P(positive|sick) P(sick) 


P(sick| positive) = Ti) Ppositive|sick)+P(noî sick) P(positive|not sick) 





1. Let Dis be the event that the person has the disease 


2. let Pos be the event that the test result is positive 


3. We know: P(Dis|pos) 


o P(Dis) = .0001; P(NoDis) = 0.9999 
o P(pos|NoDis) = 0.01; P(neg|NoDis) = 0.99 
o P(neg|Dis) = 0.01; P(pos|Dis) = 0.99 


4. What we want to compute is P(D|T), we use Bayes' rule: 
o P(Dis|pos) = P(pos|Dis)P(Dis) / P(Pos|Dis)P(Dis) + P(Pos|NoDis)P(NoDis) 
0.99 x 0.0001 
(0.99 x 0.0001) + (0.01 x 0.9999) 


= 0.0098 


This means that there is less than 0.98 % chance that the person has the disease. 


Bayesian inference for urban flow predictions 
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Sample of 50 50 





NonFan 
(40) 





6 Twilight Fans + 2 non Fans = 8 watched 
P(F|W) = 6/8 = 0.75 





P(fan) = P(A+) = 0.2 
P(nonfan) = P(A-) = 0.2 


P(watched) = P(B+) 
P(watched) = P(B-) 


P(watched given fan) = P(B|A) = 0.6 
P(watched given nonfan) = P(B|A-) = 0.05 


P(B) = P(B|A)*P(A) + P(B[A-)*P(A-) = 0.16 


50 
A+ A- 
(10) (40) 
0.2 
0.8 
\ 
B+ N N 
N B- B+ \B- 
0.6 N 
edn 05 0.95 N 
0 2*0 6 0.4*0.2 0.95*0.8 
Sen 0.05*0.8 


P(AJB) = P(BIA) * P(A) / P(B) = (0.12*0.2)/0.16 = 0.75 








1% have it Test says "Yes" Test says "No" P ( B | A) P ( A) 
Have allergy 10 8 2 P(A | B) = P(B) 
Don't have it 990 99 891 
1000 107 893 P means "Probability of" 
. | means "given that" 
e Ain this case is "actually has the allergy" 
Really ZO Test Accuracy e Bin this case is "test says Yes" 
Allergy? 80% Yes ----- > 1% x 80% = 0.8% P(A)P(BIA 
1% Has = = P(A)P(BJA) : ai bare A) 
eat’. Gato > 1% x 20% = 0.2% 
fo) 
09% A N gino > 99% x 10% = 9.9% the probability of having the allergy 
99% Not A . ne 
erea EEE > 90% x 90% = 20.4% given that the test is positive 


= 0.01x0.8 
~ 0.01x0.8+0.99x0.1 
= 0.0748... 


